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ABSTRACT 

The Borough of Havering in London^ England undertook 
a project to devise and implement a computer aided learning program 
which involved teachers in the proauctlon of materials and provided 
students with a flexible , individualized systain of instruction. 
Hattrials were developed^ and three trials were conductedi (1) a 
program and eguipment test; (2) a test of the effectiveness of the 
routing algorithm; and (3) a comparison vith conventional teaching 
techniques. This report describes the background of the project, 
outlines the tests conducted, discusses project cianageBient^ and 
summarises the results of the project's initial years of operation. 
It concentrates on tha organizational and management problenis that 
were encountered^ on attempts to assess the success in reaching 
educational objectives^ and on the possible impact of the finaings on 
te^aching method and curriculum development. 
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PREFACE 



Upon my earliest excursions into local governmenti I 
was instructed by a cynical but respected senior 
officer of the council to ensure that my reports 
had the following qualitiasi- 

1) h readable first and last paragraph. 

2) Sufficient volume of paper to justify 
the proposed expenditure. 

3) A number of tables and graphs to prove 
that the matter had been given some 
thought. 

In practice^ a more sane reporting policy is required. 

It is not my intention, in this Final Report/ to 
produce weighty tomes of evidence of our labours, 
for thio level of documentation already exists and, 
in accordance with instructions; has been deposited 
complete with packing cases with both the Social 
Science Research Council and the Council of the 
London Borough of Havering. What is more, the 
research team believes it is right and proper that 
projects should be documented thoroughly and 
completely. It has been our policy to document 
our work continuously and submit reports from time 
to time to our Advisory Steering Conmittee rather 
than leave the dpcumentation of our work to be donij 
in a flurry of activity during the last three or 
four months of the project. The result of this 
is that we are now in a position where we have said 
It all once before. This report, therefore, is 
more in the nature of a discussion of the project 
rather than a technical presentation of Its results 
and is concerned with organisational and management 
problems that we believe we have successfully dealt 
with as well as attempts to assess the success we 
have achieved in reaching our initial objectives and 
to assess the possible impact of our findings on 
teaching method and curriculum development. 

Appendix 1, however, does contain a statement of the 
complete documentation of the project and brief 
synopses of the contents of each document. 

My final and most pleasurable task in this preface 
is to record my appreciation of .the continuous and 
uiif ailing support of the many parties, without whom, 
this project would have been diminished in stature 
and quality. 

Firstly, to the London Borough of Havering , its 
Cornell, Education Committee and Director of 
Educational Services for an act of faith in embarking 
upon a journey along untravelled paths and for their 
continued support and guidance throughout the journey. 



Secondly^ to the Social SciencG Research CouncAl 
and its officars, not only for their funding but 
also for thoir continued and active support and 
their adaptability in taking a systein of 
administration principally orientated towards 
the Universities and moulding it so that it 
became responsive to the demands of working v/ith 
local government. 

Thirdly^ to our Advisory Steering Committee under 
the Chairmanship of Prof* Annett of the Open 
University p This has been a vital sounding board 
for ideas; an eKcellent forum for resolvirig 
difficulties arising from conflicting interests 
and a very real help in time of trouble. 

Fourthly^ to the teachers and heads of schools 
in Havering where we have not only been allowed 
to gain practical experience in using our system, 
but v;e have been positively encouraged to apply 
it and have received a tremendous level of support 
in designing the system. 

Fifthly, to an often forgotten body of people, 
from all walks of life, and to the Qrganisations 
who originally sponsored the Royal Liberty School 
Computer Project and thus laid the foiindations 
from which this research project and many other 
activities have grown. 

Finally, I wish to thank the staff of the project, 
both the past and present for the very real contribution 
they have made to the design and the quality of 
implementation of the system. I hesitate in the 

circumstances to single out any particular parson 
for special mention, but I know all my colleagues 
will wish me to make an exception in the case of 
Keith Lovatt, whose strong guidance and s^und 
intellect have done much to mould the final form 
of the CA.L. system and the project. 



W,R* BRODERICK 
APRIL, 1974. 
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INTHOIJUCTION 



There are two characteristics of this projact 
which might have appaared Gxceptional to the 
S* S .H, C • when the original grant application 
was made in 1969 . 

Firstly, the rGSGcirch was to be conducted in 
a school and under the auspices of a local 
education authority , Secondly , the 
project tended to fall in the category of 
applied research rather than pure research 
in education i We were positively encouraged 
by the fact that the S.S.R*C, recognised 
that applind research could be no less 
worthy nor intellectually demanding or 
stimulating than pure research. 

Our reporting policy has been one of 
continuous reporting throughout the period 
of the project. The computer system has 
been docuiTiented to the professional 
standards which are used throughout the centre 
and is completaly described in various 
levels of detail in the documents listed 
In Appendix 1. The Educational Material 

produced as part of this research project, 
is now in its final proven form and again 
is detailed in Appendix 1. The three trials 
we have conducted with the system have each 
been reported, and the results are given 
coinpletely in three reports on the 
Development of Educational Material (see 
again Appendix 1), 

This report, for clarity, contains brief 
descriptions of the computer system and 
educational material, and the three 
experiments we have conducted, but it 
focuses its attention upon the results of 
this research and the methods and strategies 
employed to achieve these results* 

Some of the original papers which led up to 
the present project were written as long as 
six years ago, against a background, where 
the then National Council for Educational 
Technology (now the Council for Educational 
Technology) was pioneering a study of Computer 
Assisted Learning in its broadest sense, in 
the face of fairly strong opposition* This 
opposition had its origins in a very narrow 
interpretation of Computer Assisted Learning 
as being on-line and interactive, I think 
in many professional educationalist's views 
eoniputer assisted learning was characterisecl 
by the turgid quality of the materials, 
presented and the inappropriatenesa of the 
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mGdiuiTt to thQ mcssagQ. Cortainly, the 
climate iuia changed but this original 
backgroiind had a fundarnental influrmce 
on the objectives of the project and 
the v/ay in which the research tearn 
approachad their taek^ , For InBtance , 
this climate influonced us to relegate 
the computer to a baclcground role and to 
ensure that it did not interfere and 
restrict what was likely to be happening 
in the classroom* Wc also felt it was 
essential to ensure that the educational 
material that was used in the project 
should be widely acceptable to teacherB 
and produced to the highest possible 
quality commensurate with the funds 
available • 

Other aspects of the background affected 
the way the project was formed and developed 
in its early days. Firstly, I was a 
practising teacher who had avoided being 
promoted out of the classroom despite 
having established the Royal Liberty School 
Computer Department and with it a pioneering 
project in computer education. I was 
sufficiently Involved with innovation to be 
concerned about planning for the future and 
in 1967 and 1968 I conducted a careful 
study of the overall role that a computer 
can play in education at school level, 
I was still sufficiently unfettered by 
years of experience in the educational system 
to hold somo irreverent views about how it 
too could be changed, and, finally, I was 
convinced of the value of managing 
educational activities as opposed to letting 
them happen. The success of this policy 
was carried forward into this project, much 
to the surprise of some colleagues at that 
time I who firmly believed that education 
should not be managed and research could 
not be managed. Two ghosts which have 
now been laid. 



2.0. 



ORIGINAL ODJECIIVES OF THE PROJECT 



The original objectives of the project were 
set out in our application to the Social 
Science Research Council dated 21st January 
1969. These are para'^phrased belowi-" 

1) To devise and implement a system 
for Computer Managed Learning* 

2) To structure material for use 
within this system. 

3) To involve teachers in the production 
of this learning material* 

4) To ensure that the Computer Managed 
Learning system implemented would be 
capable of being run on the types 

of computers already available to 
local authorities for their 
conventional data processing tasks, 

5) To provide a flexible and 
individualised system of learning, 

6) To make the system self improving 
and incorporate machine Intelligence 
techniques. 

7) To investigate the cost effectiveness 
of the system* 

8) To conduct an experiment comparing 
the effectiveness of this system with 
that of conventional teaching* 

All but one (No, 6) of these objectives have 
been achieved, although there have been 
difficulties and set-backs. The Implications 
of these and their effects on the way in 
which the objectives have been achieved are 
discussed in this report* Howeverr in order 
that the discussion can be read within the 
framework of the system we have devised^ it 
is necessary to describe the system a3 it now 
exists* These are outline descriptions and 
are included here for ease of reference p to 
understand what we set out to do^ and tha 
educational context in which it was set. 
This is described in the next few sections. 



2*1. The System Investigated 

The system being investigated relies upon the 
Computer to select for each individual student 
a work assignment from an analysis of stored 
information and of data about the student's 
abilities and attainments* The student 
works on this task and returns to the computer 
a completed questionnaire relating to the task 
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he has completed. This is analysed by 
the computer system and the most 
appropriate nsKt assignment is selected 
. for the student. Information about the 
student's progress is collected for the benefit 
of the teacher as also is information about 
the ef f ectiveneBs of specific sections of 
the learning course. The student's work 
assignments are in fact modules of a 
completely structured course of instruction. 

The use of the coniputer to control the 
student's path through a series of 
programmed tasks allows the logic of 
the system to be fle^clble, whereas with 
conventional teaching machines or programmed 
learning texts logic has to be defined from 
the outset and the conventional system is 
less adaptive therefore to individual 
circumstances* Further, the more passive 
systems are less capable (for practical 
reasons) of continuous updating and 
modification, whereas any particular 
learning task in a computer based course 
can be revised as necessary* 

In practice the system works along the 
following lines 

1) All students are given an Initial 
booklet defining a structured 
lesson f including instruction^ 
practical work and diagnostic 
tests, 

2) Students work through this booklet , 
record on a form their responses 

to particular questions as 
soon as they have finished the 
booklet. 

3) At the end of each lesson, these 
response forms are collected and 
transmitted to the Cornputer Centre 
by a courier, 

4) Upon arrival at the Computer Centre 
the information received from the 
students is processed and analysed, 
then, and depending upon an analysis 
of these responses, the computer 
system will produce for the student 
the most suitable selectipn from 
its library of possible lessons* 

This inforTnation will be transntitted 
to the school and the student during 
his next lesson will proceed with 
that material • Thus^ repeating 

steps 2, 3 and 4 above. 

9 
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EKLC 



The Research Team chose to produce teaching 
material for teaching Respiration and 
Photosynthesis, as required by most third 
year school biology courses* The method 
of teaching chosen is based fundamentally 
upon the Nuffield apprQach/ howeveri some 
significant alterations have been made. 
In parallel with the production of this 
teaching material, the t&m, have produced 
a suite of coinputer programs to operate 
the Computer Aided Learning system. 
The developing system has been tested in 
a series of three experiments involving 
more than 700 children and 14 teachers in 
15 schools. These ranged from top to 
bottom of the ability range in both graminar 
and secondary modern schools. 
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3.0. 



THE COMPUTER PROGRAMS 



The Computer Programs for this system are 
written in Hewlett Packard Algol to run on 
a Hev/lett Packard DOS-M Operating System 
computer in 2 4K of core. They require 
a control teletype^ a cartridge disc uniti 
line printer^ photo-reader and a paper 
tape punch* 

The programs can be divided into three 
suites , 

a) The Pre-Run Programs 

These programs set up files 
to be used during the course. 

b) The Course Prograrns 

These programs perform all the 
day to day operations on 
the system, 

c) The Post-Course Programs 

These programs provide final 
reports^ summary and statistical 
Information for research and 
other purposes* 



3*1, Pre-Run Programs 

The pre^run programs create four sets of 
information files. Information 
describing the Computer Aided Learning 
Task (CALT) and courses is input via 
punched paper tape and forms the NODE 
filet This file is used in turn 
to form a SYSTEMS PROFILE INDEX and 
various suunmary files* The 
description of the class, supplied by the 
teacher forms the CLASS DETAILS file* 
Finally, the students • descriptions form 
the STUDENT PROFILE, These four files 
are then used in conjunction with the 
Course Programs . 

The diagram below illustrates this process. 
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The Course progranis use the flies created 
by the Pre Run programs plus other 
information supplied by students and 
teaehars and computer operators , to run 
the Computer Aided Learning system on 
a day to day basis. 

S;tudent ^sv/er Sheets are received at 
the Computer Centre, punched into paper 
tape checked manually and then inpiit to 
the computer via a RESPONSE INPUT program. 
Simultaneously, any teacher allocations or 
modifications are converted into paper tape 
and are used to form the TEACHER ALLOCATION 
files. These files are then used as 
input for the ^ARKING program. Onm 
of the files created by the Marking Program 
is then used by a program which updates 
the student and system profiles and it is 
these files along with the Teacher 
Allocation files i the Node file and the 
Class Details file which are used for 
input to the ROUTING ALGORITHM^ v/hlch 
in turn selects the niost appropriate 
next lesson for the student and creates 
as its output a disc file., This Disc 
file along with such data as date and 
class number are used by th^ output 
progrMS. The output programs prov^ide 
daily and weekly output thus, completing 
the task of the Course programs* 

This process is represented dlagraimnatically 
below. 
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PIAGIUMMATIC REPRESENTATION OF COURSE PROGRAMS 
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3.3. Post Course Programs 



The Post Course Programs, which are used by the 
research staffs to provide suimtiarles describing 
the behaviour of the system and statistical 
Information and also the final reports for 
schools call upon a number of disc files* 
Data can be taken from them in differanc 
combinations for different purposes and they 
provide a variety of forms of output for 
later study and analysis, 

The diagram below illustrates this activity^ 
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4.0. THE OPERATION OF THE SYSTEM 



All the students upon entering the ^sy tern 
. arc qivun a pre-^course test which is used 
to creacQ a profile of Information about 

a) the student* s ability and 
aptitudes^ 

b) his pre^knowledge of the 
learning materials and 

c) the pre^requlsites for 
following the course. 

This test, like all the others in the 
course, is marked by the computer which 
simultaneously updates the student* s 
profile. Subsequently, using a 
"routing algorithm" the computer selects 
the next lesson for the student* At 
this stage it prints out 

a) for the student, information 
about what he has to do during 
his nsKt lesson 

b) for the teacher, information 
about what the whole class is 
doing and 

c) for the laboratory assistant, 
what apparatus needs to be 
prepared. 

This output from the computer is taken 
to the child's school by courier^ the 
child does the lesson as directed. At 
the end of the lesson the student answers 
(usually in written form) some 
diagnostic and predictive questions on 
what he has learnt during the lesson. 
These answers are collected by the teacher 
and returned to tha Computer Centre by 
courier* They are then punched Into 

paper tape, marked by the computer and the 
cycle continued. 

This situation can be shown diagramiiatically 
in the figure below. 
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The role of the teacher in this system is 
different to that In "conventional" teaching. 
Much of the instructional role is removed 
and replaced by the need to help individuals 
with' their specific difficulties. The 
computer provides the teacher with infoTOatlon 
to help him manage his classroom and the 
students learning within it* Before a 

lesson the teacher is given^ printed reports 
on the studentfs achievemant to date in the 
course, on the work allocated to them for 
the next lesson and on what the laboratory 
technician and teacher need to prepare for 
the next lesson (including equipment and group 
tutorials respectively) * * 
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5.0, WRITING EPUCATIONAL MATERIAL 



The first choice which we were faced with 
was that of the subject area. This had 
to be an area which was self contained 
and reasonably structured. Indeed^ 
the policy adopted was to choose the best 
person available to be the Educational 
Analysts then work in the subject area 
in which that person had had considerable 
teaching cKperience. This led us to 
a choice between Biology and Geology - 
Biology was the obvious choice, becaus.^ 
most children in Havering secondary schc^ls 
study Biology and, therefore/ we would 
have a potentially large population to use 
the systeiti. 

Having decided on the subject area^ contact 
was e&;tablished with the teachers, who would 
be involved (c.f, objective 3 in the 
original objectives of the project). They 
were called upon to help us select both 
subject area within Biology and consequently 
the age of students with whom we v/ould work- 
This initial involvejment led us 'naturally 
to using the teachers to help us ins-- 

a) Specifying of the objectives of 

the course* 

Reviewing the written material produced 

yalidating the material by testing 
it in their classes , 

Assisting in conducting the trials 
of the system. 



b) 
c) 

d) 



5.1, Respiration 

It was agreed initially that we would produce 
material for the whole of the ability range, 
but it was decided to produce material for 
the most able students first. We agreed 
the course objectives with the teachers 
concerned for a five week course in Respiration 
for third year students based upon, but not 
slavishly following, the Nuffield Biology 
Curriculuni. The course objectives were 
agreed upon. These objectives fell broadly 
into four groups 

a) Those coiiOTion to all student ability 
levels. 

b) Those specifically for most able 
students. 

c) Those specifically for middle ability 
students . t a 



Those for least able students. 



The differences between b^ c and d reflect 
both differences in content and approach. 

These were ordered in a hierachical manner usin 
a form of Gagne backward analysis* At the 
same time test questions were produced to 
test the achievement of these objectives. 

The next slage was the production of 
Computer Aided Learning Tasks (CALTs) 
which were the teaching materials with which 
the students would work,, and which would 
provide the students witR. the Information 
they would nee#, to answer the test questionB, 
The first drafts of these inaterials were 
assembled into Gourses and then tested in 
schools, independently of the computer systein. 
This operation was Repeated to produce 
suitable materials 'far -the middle ability 
range and the least able students and when 
the whole course had been prepared, it was 
implemented on the computer system. 



5.2* Photosynthesis 

For reasons given later in the report, it 
was not possible to follow the same procedure 
for the Photosynthesis material and the 
opportunity was taken to investigate 
different ways of producing material for 
this system. In each case the educational 
objectives were agreed with teachers as 
before and test questions defined by an 
Educational Analyst. A team of six authors, 
under a Chairman,^ was 'then commissioned to 
write teaching materials for the most able 
students. This approach resulted in the 
production of tested arid validated materialr 
some of which was of very high quality. 
Because the team of authors were working 
independently, the style of the material 
lacked coherence and it was decided to 
adopt a different approach in producing 
material f or th^ ^middle range of students* 

In this case to provide coherence, two teachers 
in the same school and both eKperienced in 
working with students of this calibre, were 
coiranissioned to prepare material for this 
course. Again, they were working with 
educational objectives which had been agreed 
with teachers and specified by the Educational 
Analyst, who had also prepared the test 
questions. These teachers wrote and 
validated their material which was adjudged 
rather more successful than that produced 



The educational material *for the least 
able students in Photosynthesis was 
written at the Computer Centre by the 
Educational Analyst, at the same time 
as the teachers in the school were 
producing the material for the middle 
range students* 

The Photosynthesis course has not yet 
been incorporated into the computer 
system because it is most sensible to 
test this in the sununer v;hen there is 
sunshine, which will enable photosynthesis 
to take place. It is expected that this 
work will be undertaken during the 
suunmer of 1974. 
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THREE E)(PERIMENTS 



The plans for the project allowed time for 
^us to conduct three eKperiments with the 
complete system. Each expariment was 
designed and timetabled to benefit from 
the experience gained from the former 
ones and give us i wherever possible, time 
to Implement any changes that were required. 

In total these three eKperimentB have 
involved more than 700 children and 14 
teachers with using this system over a 
period of eighteen months* These 
eKperiments are described and discussed 
fully in the Second and Third reports 
of the Development of Educational Material , 
which are referred to in Appendix 1. 
Appendices 2 and 3 contain short descriptions 
of the student samples, experimental design 
and results of experiments 2 and 3, 
This has not been done in the case of the 
first experiment because the objectives of 
this experiment were mainly operational 
rather than educational* It must be 
emphasised at this stage that the descriptions 
following refer only to the principal 
objectives of the eKperiments and that much 
other valuable information has been collected. 



6.1. The First Experiment 

The objectives of this experiment were 

a) To see that the system worked. 

b) . To discover what the operational 

difficulties werei both in the 
Computer Centre and in the classroom. 

c) To overcome these dif f iculties • 

d) To enable us to measure the cost 
of this system and, 

e) To find out whether in practice 

the Routing Algorithm did individualise 
learning. 

Finally ^ It was intended to use this material 
to indicate where remedial CALTs were 
required and to produce these before the 
second experiment. 

Results of the first experiment 

As a result of this eKperiment, we discovered 
that the additional operational cost* of 



*by additional operational cost we mean that 
cost of the courier service, computer time 
and personnel to operate the CAL" system, 
R. & D. costs are excluded from this figure. 



running the Coniputar Aided Learning System 
was 25p. per student hour and that some 
modifications were required to our 
operational procedures at the Computer 
Centre to simplify the use of the systemt 
There were some incompatibilities between 
our prograins and the operating system 
which led to an (apparently) random loss 
of data^ Finally, it was decided that 
the research staff were not needed to 
provide the courier sarvice and that we 
could use one of the local authority's 
couriers for this purpose. This trial 
did yield educational data which we were 
able ttocombine with that from the second 
experiment to increase our sample sizes. 
Because no two students followed the same 
course through the learning material we 
were able to conclude that the Routing 
Algorithm did individualise instruction. 

6.2. The Second Experiment 

The purpose of this eKperiment was to explore 
the effectiveness of the Routing Algorithm 
and to compare this with manual allocation 
of CALTs on the basis of pretest scores. 
One of the objectives of this experiment 
was to investigate whether the facts 
discovered from the first experiment i 
namely, that no two students had taken 
the same route through the material^ made 
any significant difference to the learning 
(as measured by the post test) of the 
students involved . 

The experiment was set up using the same 
CALTs as in the first experiment^ with the 
addition of the remedial CALTs, The 
experiment was blind, in that neither the 
teacher or the student knew who was being 
routed by the computer or directed along 
a pre^selected course of material which 
was chosen as a result of the performance 
in the Pre--Test for the course. This 
experiment was designed to i>e a severe 
test for the Routing Algorithm, but one that 
was more than justifiable as the algorithm 
involved a significant amount of computing 
time and was a costly part of the computer 
system to produce , 

Results of the second experiment 

Despite the fact that we ran it with a small 
population, it was considered to be a worthwhile 
experiment.. We were able to conclude from 



this eKperiment that the routing was of 
significant value to the learning of 
s-tudentB in the lower ability ranges^ 
but that these results were only 
significant at the 51 level. In any 
CQurse there is a limit on the upper 
and lower bounds of material provided. 
It may be argued that able students did 
not gain signficantly through the routing 
procedure since they needed the upper limit 
of the material incorporated in the system. 
Therefore, the routing procedure could not 
help them. We also die covered that we 

ware able to use,. the local authority's 
courier service .for despatch and 
collection of materials and that the cost 
of running the system could be reduced to 
21p» per student hour. As a result of 

this costing it was agreed that soine 
alta rat ions would be made to the computer 
programs and to the routing algorithm in 
particular* These alterations were 
designed to enable the system to function 
wore quickly, and thus more cheaply. 



6.3t The Third E^periinent 

The principal objective of this experiment 
was to compare the system with conventional 
teaching and to try to reduce the 
operational costs etill further. There 
are fundamental difficulties^ which are 
documented elsewhere # about both the form 
and validity of any comparison with 
conventional teaching. An experiment 
was designed because it was recogiiised 
that this comparison had to be made despite 
the impossibility of conducting a really 
rigorous eKperiment. It' was decided, 
however/ to weight the eKperiment against 
the C,A,L. system. The teachers involved 
in this trial knew the purpose of the trial 
and were chosen because each was teaching 
in the same week two different classes of 
the same age and ability range. ..It was 
agreed that the teachers would have complete 
access to all the CiA.Li materials^ and that 
the group they taught first in the week would 
be taught using the C,A,'L, system, Irrtmediately 
afterwards they would teach their second 
group conventionally. The teachers knew 
v?hat the objectives of the course were and 
recognised that they were not competing against 
the computer but seeking to discover whether 
teacher plus computer was more effective than 
the teacher alone. 
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Results of the third eKperiment 



The conclusions that v/e WBXm able to draw 
from experiment three were that the teacher 
with the Computer Aided Liaarning system 
was more effective than the teacher 
alone and this result was significant at 
the 1% level. We were also able to 
discover that the cost of operating the 
e.A.L* system could be reduced to 19p. 
per student hour* 
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some' problems encountered 



7*1. Variety of Data Form at 

The first major problem that we encounterecl 
was the incredibiy wide variety of formats 
of data that would be needed to Input the 
student and course descriptions. These, 
if they had been coded separately, would 
have involved a vast ajnount of progranuning 
time and effort an^ as this design work 
was undertaken at the early stage of the 
project they would have been very susceptible 
to the need to be changed during the course 
of the project. In order to preserve 
flexibility and reduce the total Miount 
of computer storage and^ programming time 
involved in the system, it was decided 
to produce a general purpose system which 
could be used widely within the C.A,L. 
system. This system which we called 
the FILE DESCRIPTION SYSTEM is 
described in detail in the document 
referred to in Appendix 1 5(b), ^ 



7*2. Computer System 

Our second area of difficulty was associatad 
with our computer system. The original 
project proposal was dependent upon the 
enhancement of an existing computer system 
with both additional hardware and software. 
In the event the computer manufacturer 
failed to supply equipment which would have 
enabled the objects of the Research Project 
to be realised and the legal aspects of 
this situation are currently receiving 
consideration. This difficulty forced 
us to apply to the Social Science Research 
Council for a supplementary grant to 
enable the duration of the project to be 
extended and additional computing equipment 
to be purchased. These events occurred 
in a hectic five month period in the middle 
of 1971, 



7.3, Algol v Fortran IV 

There was pressure on us when changing machine 
to change from Algol to Fortran IV, This 
was a change which was resisted at that time 
becausei * 

a) All our expertise and experience 
was in Algol , 

b) Algol was a good language for 
describing algorithms, and that 
is how we saw our work. 



c) The program translation time 
from one version of Algol to 
another would probably be 
less than from one version 
of Algol to Fortran IV, 

d) We v/ere handling compleK 
logical and data structures 
which were more sympathetic 
to programming in Algol 
than in Fortran. 

e) Transferring both the work of 
this research project and the 
work of the Computer Centre 
from one machine to another 
in the period of less than 

two months (and with no overlap 
of machines) gave us almost more 
work than we could handle at 
that time . 

Despite the above justifications^ present 
pressures on us regarding the future 
use of the system and the increasing 
interest, not only within the National 
Development Programme in Computer 
Assisted Learning, but more widely, 
in the transportability of Computer 
Aided Learning, makes us feel a little 
sad that we were unable to recode In 
Fortran IV at that time. I do not, 
however, regard our failure to have 
done so, even'* with the benefit of 
hindsight as a major failure. 



"7*4, Staff Illness 

Our next and entirely unfors^eable problein 
was the serious illness for nine months of 
the Educational Analyst on the project. 
Whilst. being absent from work, he continued 
to make a significant impact on the thinking 
of the project throughout this time and 
instigated the eKpariments referred to earlie 
using different Ways of writing educational 
material* The' fact that this illness was 
turned to advantage, is a real tribute to 
him and to his ability to work to the highest 
possible standards under the most trying 
conditions. Our sorrow at our colleague's 

Illness was tempered by the fact that it 
occurred at a time when the project team 
were concerned principally ^ with re*writlng 
the programs for the new computer system 
and apart from limiting the volume of 
educational material produced, no other 
significant effect was made on the project. 
On balance we feel that this lllnesa was 
the main reason why less educational material 
was produced than was originally intended. 



The lawl trunjor difficulty to bo discussed 
here, ia that of o valuation. Without 
miy doubt at all, this haa been the 
yroatust fKhicntionr:! 1 ijroblarn that, wo 
havo tackled nnd the most difficult 
one to aolvG. We havn achiovad a 
fiolution which is acceptablo to all 
the parties concGrncd (although not 
nocossarily an optinial solution). 
Appendix 4 outlinGB tliose characteristics 
of the ays tern that makes evaluation 
eepocially difficult in our circumstances. 



of our problems have arisen due to 
a conflict of intereato. We set about the 
task of uvaluation by posing ourselves 
the quer,tlon "who requires us tu evaluate 
this system"? To which we answered 
"the public, our Education Commit tee and 
Council, the G.S.R.C., our colleagues and 
teachers". We then attempted to pose 
the questions to which these people would 
want answers. We often asked them 

directly. Frankly, this was not a 

very illuminating expGriance. We came, 
ultimately to the conclusion that our 
evaluation had to satisfy the S.S.R.C., 
in terms of critical analysis, our local 
authority, in terms of enabling them to 
make decisions whether to use the system 
in the future or not and our teachers 
in terms of whether they v/ould find the 
system helping them achieve their 
educational objectives. We thus came 
to the conclusion that our evaluation must 
bo descriptive, qualitative and quantitative, 

We are still restricted in the evaluation 
we can do by the fact that the research team 
is small and that our evaluation can only 
be short term because 

a) In a project of limited duration the 

time that can be spent on evaluation 
is short and the depth to which 
it can be carried is limited, 
(a problem which is exacerbated 
by the fact that the expariments 
have to be done at the end of a 
project of this nature). 

b) Our populations are relatively small. 

c) Any long term evaluation involving 
a small amount of educational 
material, and therefore the impact 
that it is likely to have on children, 
is unlikely to show effects which will 
be discernible at long range through 
the background haae created by |^1 
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iho ntlinr jnfluanauB on tho child, 

Tharo ain no Bimplo anywurB to thia problc 
but Btopo havo buun taken to onsuro that 
in our futur^' v/cjrk wn arc supportGcl by 
outBide aval ua tors and consultants to 
guldo UB iri thlH Bide of our work on 
future projects. 
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8.0* PROJKCT MANAGEMENT 



It W£is our doclarGd intantion to demonstrate 
that wo could manage rt^Bonrch as wg]1 as 
learning. The fact that we have 
achiovod our objectives within the time 
scalD specified and within the costs 
specified initially^ will be evidence of 
this , 

The first stage in the design of the 
management system for our project was 
the establishment of an Advisory Steering 
Committee ; which would oversee the work 
of the project. This committee which 
consisted of representatives of the 
S.S.H.C, the London borough of Havering, 
the Research Team and our two Consultants, 
has had its role described earlier. 
Because this committee consisted of a 
group of people who were themselves 
interested in the achievement of the 
objectives of the research they played 
a positive role in the management 
and development policies of the project. 



8*1. Timetable 



When the project was originally proposed a 
timetable for it was drawn up. This 
set out in three monthly periods what was 
expected to be achieved during that period. 
Despite the difficulties listed earlier, 
that timetable has been adhered to 
throughout the project. The purpose of 
the timetable (which was supplied to every 
job applicant and certainly scared a few 
away) , was to foct\s attention on achieving 
a realistic set of objectives in a 
carefully laid down time scale and for agreed 
costs. All the staff working on the 
project recognised the need to adhere to this 
timetable or the project would fail* 
Secondly, all the senior staff of the project 
were responsible and accountable for 
achieving (or failing to achieve) the 
objectives in each period* This responsibility 
focused their attention on the need to 
achieve their objectives and their personal 
role in doing ao. 




8*2* Financial Control 

Our organisational management has been 
considarably helped by financial management. 
The professional assistance of the Borough 
Treasurer of the London Borough of Havering, 
is gratefully acknowledged, who guided us 
through the initial costing of the project 
and set up a cost coding system, vihich enabled 
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us to analyse expenditure in detail. 

The S.S.R.C.'s system for analysis of 
exponditurc was not found to be especially 
hQlpfui for our own internal expenditure 
analysis and prediction and to ensure that 
runaway expenditure did not take place, 
annual allocations from the grant were 
made under various cost headings and the 
rate of expenditure was monitored to 
ensure that it fell within the limits 
laid down. Once the system had, been 
established and due to the thorough 
budgetting that took place initially. 
It was not demanding upon the Proiect 
Director. 



8- 3' Interfaces between activif-i 



es 



es , 



In C.A.L. research much effort has been 
concerned with producing author language 
We decided not to do this, but to difine 
in the early stages of the project 
an interface between educational material 
and the computer system. This was done 
tor four reasons :- 



1) 



To help us manage the project. 



2) To bring the research team together 
and focus their attention on 
common problems. 

3) To ensure that the system was 
generalised right from the 
beginning. 

4) To ensure that an Educational 
Analyst could write his 
instructions for the computer 

in a language which was familiar 

to him, and independent of 

the computer system being used. 

we have no doubt at all that this interface, 
which allowed the two halves of the project 
to evolve separately (and out of phase at 
times) played a significant part in our 
achieving our objectives. Defining 
the interface between different activities 
in a research environment at the beginning 
of the project can help in the achievement 
ot overall objectives. Despite the fact 
that this IS a difficult and time consuming 
task. It does focus everybody's attention 
towards the actual problems to be solved. 
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8,4. Some cansequences of managament 



Somo of the rGSults of the careful managemGnt 
of this project have been quite discernible 
over the five year period since its inception . 
The attitude of the local authority and 
its senior officers has changed from one 
of healthy suspicion to one of support and 
respact. The teachers involved with using 
this system have come to know that they 
can rely both on the system and the research 
team, and have eKpreSBed the opinion that they 
wish to continue using the system. This 
is evidenced by the fact that since the second 
trial we have had more people wishing to use 
the system than we have been able to 
accomniodate , 



8.5* Plannin g future project 

Reference is made later in this report to our 

view of the future research and development 

work needed with this system. It was 

necessary for us to apply to the Social 

Science Research Council for a research grant 

to continue our research before the 

termination of this project. Ideally^ in 

the opinion of the research teamf an 

extensive research programme was the next 

loy^ical BtsrgBf but in view of the large sum 

of money involved ^it was decided to apply for support 

for a two year project and review the situation 

towards the end of that stage. This application 

(Refi HR 2798/1) was made on 27th September 1973 

and was successful. However, it has brought 

home to us the difficult management situation 

where one seeks to keep together i for an 

extended period, a research team which embodies 

a unique set of skills and experience. 

Because we are obliged to give these staff 

three months notice we have to have a decision 

as to whether or not we will be able to continue 

our work three to four months in advance of 

the end of a project. Because it takes roughly 

twelve weeks for the Social Science Research 

Council to consider a project costing up to 

£50,000/ this means that an application for 

further research must be submitted six months 

before the end of the current project. It 

takes nearly two months to design the project 

and have it approved by our local authority 

before making an application to the S.S.R.C. 

This constrains us to preparing our application 

for further research some eight months ahead 

of the end of the current project. This 

problem is exacerbated if the project is going 

to cost more than £50 ,000, when an application 

has to be submitted to the S*S.R,C, in August, 

in which case the result is not made available 

until the following April. Looking ahead 
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in our own Ckibq, this prGsents a ridiculous 
situation in that if wg wish to continue 
with our rosoarch beyond the next two 
ycors we may nciad to submit an application 
to the S.S,R*C. after only four months work 
on the next phase starting in May. 
I do think that in areas where continuity 
of research is desirable the deve; opment 
of a more floKible policy in this matter 
could be to the S.£.R,C,*s advantage* 



8p6, Use of artificial intelllgenca techni ques 

When th^ original project pronosal was written 
we anticipatf3d that benefits could arise 
from making the system "self improving", by 
including machine intelligence techniques 
in our programming* Such techniques have 
not been included in our work to date 
because we are not yet convinced that there 
are any that could make a contribution 
towards refining or improving the effectiveness 
of the system. In planning and designing 
the system there were many techniques and 
characteristics which v;e considered building 
in, Thp criteria that we used were 

a) That we had a very high expectation 
that they would make a significant 
contribution to the overall 

af fectiveness of the systemp 

b) That we could predict what difference 
they would make to the system. 

Despite a study of machine intelligence literature 
and systems we never were in a position where we 
had a high expectation that it would make a 
significant contribution to the overall 
effectiveness of the system^ nor did we feel 
that v;e could predict what contributj.on it 
might make. This area is^ therefore i left 
open as an area of future investigation and 
we tend towards the opinion that this may be 
more appropriately conducted in one of the 
artificial intelligence research groups than 
in this Centre. 



8.7. Copyright 

A study has been made during the course of 
the project by the Borough Secretary's 
Department of a way in which the laws of 
patent and copyright protect material produced 
under the auspices of this project. The 
thoroughness of this work is a reflection 
upon the painstaking way in which it was 
undertaken by the Borough Secretary and its 
usefulness has been considerable, not only 
to ourselves and the Advisory SteGring Committee, 



but also to various groups outside this 
project who have faced the same problem 
in a very similar context* 

8 * 8 * Commercial Exploitation 

During the autumn of 1972^ as soon as it 
had been demonstrated that the Computer 
Assisted Learning System worked^ the 
research team and the Advisory Steering 
Committee decided that within the 
terms of our contract with the Social 
Science Research Council we ought to 
consult the National Research Development 
Corporation about the commercial 
exploitation of our system. We 
accordingly approacheQ the N,R,D,C. 
However, there was a very considerable 
delay on the part of the N.R.D-C. in 
responding and when ultimately they 
did so their attitude proved negative. 

As a result of this delay on the part 
of the N.R.D.C, we are not as far ahead 
with the exploration of the commercial 
exploitation of the C.A,L. system as we would 
have hoped. However, it was discussed at 
the penultimate meeting of the Advisory 
Steering Committee and it was agreed that 
a consortium approach would probably 
be the most satisfactory. The establishment 
of such a consortium is now being studied 
and preliminary contacts are being made 
with appropriate industrial concerns , 



8,9. Relationship with other bodies 

The staff of the research project have travelled 
extensively both in the United Kingdom and 
in Europe. This time has been well spent 
both in terms of disseminating information 
about other projects. The travel within 
the United Kingdom and abroad has enabled us 
to study how different forms of Computer 
Assisted Learning are operated in different 
educational philosophies. It has given 

the research team an overview of the problems 
involved and focused their attention on 
areas of need and application for computer 
assisted learning. There is no doubt that 
the information obtained from these visits 
has had a real effect on the total concept 
of the system and' the way in which it has been 



implemented. It has also donG much to 

focus our attention on different aresis of 
futurG application of Computer Managed 
Learning in general. 

The research team have produced a nurnber of 
internal papers on their work^ thesa papers 
form the Educational Reports listed in Appendix 

Wo have been restricted^ however, in how much 
we have been able to publish externally 
because of the necessity of keeping aspects 
of the work confidential^ and by our 
responsibilities to the S.S,R.C, in terms of 
the commercial exploitation of the results 
of the research, Thus^ only those 

aspects of the computer system which are non- 
specific (e.g. File Description System) have 
been published along with an overview of the 
system in "Aspects of Educational Technology'', 
A paper on the educational aspects of the 
research is in preparation and will be published 
during 19 74. Further papers on the system 

will be submitted to the World Conference on 
Computer Education in 19 75, 



9.0. WHAT NEXT? 



A computer aided Learning system which teaches 
siK weeks Biology to third year students in 
the London Borough of Havering and does nothing 
else is unlikely to justify the energy and 
resburces that have gone into its design and 
implementation. We must^ therefore , study 
areas in which it can be applied to real 
and identifiable^ educational problems as 
well as those which can be studied for 
research purposes. 



9 • 1 . Identification of other subjects 

The material for teaching photosynthesis 
is going to be incorporated in the system 
this summer and it is expected to become 
publicly available in the autumn. Plans 
are in hand for implementing a teacher 
Independent course in the computer language 
BASIC before the end of this year and 
consideration is being given to developing 
Computer Aided Vlth form courses in either 
Modern Languages or Mathematics for the 
non--specialist. Finally^ a Design Study 
in Computer Assisted Learning has just 
been completed under the auspices of the 
National Development Programme, to investigate 
a curriculum model that can be used with a 
computer system to provide Remedial Reading 
for seven year old students. 



9*2* Availability of the computer system 

At the moment the computer programs will run 
on Hewlett Packard Disc Operating System 
computers with appropriate core storage and 
peripherals. These programs will have to 
be translated on to IBM, ICL and Honeywell 
computers if they are to become more widely 
available to local authorities and universities. 
Hopefully, other institutions using the system 
would also produce teaching material for use on 
it. Thus, we see a long term prograimne of 
dissemination as being a possibility. This, 
however, will only be successful if the system 
meets real educational needs and can 
demonstrably make a real contribution to the 
learning of the students who use it. 



9t 3. Operational Costs 

The present level of operational costs is 
about 101 or less of the costs of further 
and higher education. It becomes much more 
significant, however, when compared with 
the costs of secondary and primary education. 



This is GspGcially true when it is recognised 
that tho cost of running this system is an 
additional cost over and above that of 
conventional teaching. Every effort must 
be made in the future work on the system 
to both increase its effectiveness and 
reduce its operational costs. Thus , all 
development work must assume a responsibility 
to ensure that it is a realistically viable 
system for implementation on a wider scales 



9*4, Future research 

In our application to the Social Science 
Research Council for our next research 
project based on this system^ we have 
outlined the areas of future research 
which we felt had already been identified. 
These are listed briefly here and are 
divided into four main areas. 

i) Routing Algorithm 

A. Investigation of different 
automatic routing methods 
for which the computer is in 
control (including different 
algorithms) . 

B. Investigation of the effectiveness 
of teacher selected routes on 

. student learning, 

C. Investigation of the effects 
of student se le cted routes on 
student learning . 

ii) Output Methods 

D. Thorough investigation of the 
form of the output to teachers 
and students * 

E. Investigation of the methods of 
assessing individual student 
progress in open ended teaching 
situations, 

iil) Teaching Methods 

F. Investigation of the effects of 
students working in small fixed 
groups throughout the course, 

G. Thorough investigation of the 
teacher taught lessons with the 
computer marked tests for 
diagnostic and prescriptive purposes 



iv) stud ent Attributes 



H, A study of the effect of different 
students learning styles on the 
effectiveness of CA.L* 

I. A study of the effect of personality 
on the choice of- the most suitable 
teaching/learning method. 

It is proposed that some of the additional research 
will be done using the biology teaching material 
which has already been written as well as 
developing other material described above , 
The advantage of using the biology material 
initUally is that we do have a considerable 
background of experience of using this course 
thus experiments can proceed without further 
delay. 

Taking all the factors into account it is 
estimated that this total foreseeable research 
activity would involve five years work for 
a relatively small team. The cost of such 
a research programne would be £106,000 at 
June 1973 prices* 
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10. 0, h SUMMi^RY OF THE ACHIEVEMENTS OP THE PROJECT 



It is appropriate, having started this report 
by studying the initial objectives of the 
project^ and having xanged through the whole 
life span of th^^^ project to ^ study of 
future dpyaiopments , to finish on a note 
which records our actual achievements to date, 

1) ^ system of Computer Managed Learning 
has been implemGnted and demonstrated 
to work on a mini computer. 

2) A system of Computer Managed Learning 
has been devised in such a way that 
it is independent of 

a) the subject matter being taught, 

b) the taxonomy of educational 
objectives being used^ 

c) and to a considerable extent 
of the curriculum model being 
operated, 

3) We know what it costs to operate 
the system. 

4) A system of Computer Managed Learning 
is possible in organisational terms 
on a secondary school working 

envi ronment , 

5) The system of Computer Managed Learning 
is acceptable to teachers in the field 
of science teaching at least* 

6) Ws have established a pattern of working 
for the production of Computer Aided 
Learning Tasks ^ which is both effective 
and useful with other subjects than 
Biology * 

7) We have established that a group of 
teachers operating with Computer 
Managed Learning are mora effective 
than the saine group operating without 
it. 

8) We have identified furthi§' areas 
of research which will refine our 
knowledge about the system* 

9) We are convinced that a Computer 
Managed Learning system can be a 
powerful test-^bed for educational 
research. o Q 



ultimately I our work and its value must 
be evaluated by the impact it makes on 
Computer Assisted Learning research and 
development both in the immediate future 
and longer term. We will be happy 
that our work should be measured against 
this yardstick * 



APPENDIX 1 

C.Aai. PROJECT DOCUMENTATION 



1) EDUCATIONAL REPORTS 

a) "The Development of Educational Material 

anT'Interim Re|3ortf" - May 19 71, " 

This report describes the educational 
work on the project* Particular 
attention is paid to methods of 
selecting educational objectives, 
the establishment of a taxonomy 
system* It includes examples of 

CAL texts* 



b) "The Development of Educational Material 
The Second Report^'"- May 19 73 

This report describes two years work 
after a) above. It covers additional 
work on the production of teaching 
material and test construction. 
It also contains work on the evaluation 
of the system, the results of Trials 
1 and 2f some discussion of these 
results and an initial description 
of the costs of the system* 

c) "The Development of Educational Material 
The Third "Report - February 1974 

This report covers the work dons in 
the third trial of the C*A*L. system 
and reports on this under the same 
headings as used in b) above* 



2) TEACHING MATERIAL 



a) Student CALTs Level 1 

Respiration for the most able group 
of students. All CALTs numbered 
1000 - 1999. 

b) Student CALTs Level 2 

Respiration for the middle range group 
of students. All CALTs numbered 
2000 - 2999. 



c) Student CALTs Level 3 

Respiration for the lower ability group 
of students. All CALTs numbered 

3000 - 3999, 
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d) 



En r i chmcn t CALTs 



For varying ability ranges. All 
CALTs nun^ered 5000 - 5999. 



e) Remedial CALTs 



These provide remedial instruction 
for all ability ranges* All 
CALTs nun±>ered' 6000 6999. 



f ) Tests 



The pre-test for the system - 
also used as the post^test. 



g) CAL Diagrams 



Students manual of diagrams for 
use- with f) above. 



SUPPLEMENTARY EDUCATIONAL MATERIAL 

a) CAL in the Classroom 

A description of teachers of the 
operational aspects of the CAL 
system* Describes their role 
and gives examples of the output 
produced by the system, 

b) Teachers Manual 

A manual to help the teacher use 
the CAL system. Advises on work 
patterns, procedures for the teache 
to override the computer, how 
to use the subjective assessment 
and guidance on the teacher 
directed tutorials which form 
some CALTs* 

c) Laboratory Technicians Manual 

A complete set of instructions to 
enable the Laboratory Technician 
tO' be of maximum assistance to the 
teachert It also contains a 
directory of all apparatus used. 
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d) Ta pe/Slide Material 



or 
Slide 


Pnr use 

with 
CALT no 


Brief Description 


S 


1020 




This CALT is a tape/slide 
tutorial on energy 
cycles , 


T 


^ 1020 


This tape is the 
commentary for this 
CALT which is a tape/ 
slide tutorial. 




4000 


1 


T 


4001 






T 


4002 






J. 


4003 








4004 
4005 




Spoken commentary 
^ of CALTb for 
poor readers. 


T 


4006 






T 


4008 






T 


4009 






T 


4018 







RESPIRATION - INTERFACE DOCUMENTATION 

This contains a copy of all interface dociiments 
prepared to put the Respiration course into 
the system. 

INTERFACE AND ROUTING 

a) Educational Interface Specification 

This manual describes in detail the 
interface between "the computer programs 
and the teaching material. It 
refers to the conceptual structure of 
the material, the types of responses 
that can be input and the method of 
expressing educational information 
in a form suitable of computer input 
4) above is an exMiple of this. 



b) Fl le De s c r 1 p 1 1 on System - W.R, Broderick 
i Margaret E, Bryant 

This document describes the flexible 
method of inputting files used by 
the CAL system* 

c) The Routing Algorithm 

This document specifies the routing 
procedure used to allocate the "most 
suitable" next lesson to students* 



COMPUTER SYSTEM 

a) The Systems Manual 

This manual describes in detail the 
concept of the system^ the structure 
of the suite of programs^ the form 
of ^ the files and the operating 
Instructions. 

b) Examples of Data and Output 

This manual provides an illustrative 
set of data for each program in the 
system to complement the descriptions 
in the Systems Manual. 

c) Program Documentation 

For each program, at least a brief 
description^ examples of input and 
output and flowcharts. 

d) Program Listing and Source Statement 
Tapes ~ 

These are provided for all programs 
in the system. 



ADMINISTRATIVE NOTES 

a) Annual Reports 

This file contains copies of each 

of the Annual Reports to the S.S.R.C. 

b) Advisory Steering Committee Minutes 

This file contains copies of each of 
the Minutes of the meetings of this 
Comtuttee. 
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9) 



FINAL REPORT 



This file is the Pinal Report on the Project 
to the S.S.R.C. 



AVAILABILITY 

Items under Headings 1 and 9 are publicly available 
Items under Headings 2 and 3 are available only to 
users of the ca.l, system, 
Other items are confidential, 
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APPENDIX 2 



TRIAL 2 DESCRIPTION 



Object 

To discover if the routing algorithm of the computer 
aided learning system is making any contribution 
to the gain in pre/post test scores of students 
using the C.A.L. System in Biology, 



Student Sample 

A sample of six schools were chosen ^ one granmiar 
school and five secondary modern schools. This 
gave a sample of 209 students 33 of which were 
from grammar school. At the end of the trial 
the number of students available for statistical 
treatment was 154 ~ 30 of which were from the grammar 
school. 



Experimental DeBlgn 

The students in each school class were divided into 
two groups according to their score on pretest 7. 
The two groups were formed by putting into group 1 
the firsts fourth^ fifth, eighth, ninth etc. students 
rank order* Whilst into the group 2 the second, 
third, sixth, seventh etc. Neither teachers or 
Btudeats knew that this subdivision had been made. 

One group of students was routed automatically by 
the computer routing algorithm. The second group 
of students was given one level of teaching material 
by the computer according to each students score 
on pretest 7 as follows:- 

Score on Pretest 7 equal or - Ability Level 
over 70% 1 material 

Score on Pretest 7 equal or - Ability Level 
over 35% 2 material 

Remainder of students - Ability Level 

3 Material 

Both experimental and control groups received work each 
week for five weeks. From the teachers and students 
position everything appeared to be routed by the 
computer* 



Results 

Results comparing the two samples complete showed no 
statistical difference in post test scores or gains 
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for the two groups (77 students in each group) . 
However, if the lower ability students are compared 
(i.e* those with a pretest score below 56% on prate 
7) then the computer routed group do significantly 
better. 



Raw Gain of Students completing pretest and post 
t e s t 7 V Who le s am p 1 e 



Experimental Control 





(Routed) 


(Non-routed) 


MEAN 


13.896 


13.143 


S.D. 


10.953 


9.608 


N - 


77 


77 



Raw gain of students completing pretest and post 
test 7 who have pretest score of 55% or below. 



Experimental Control 
(Routed) (Non^routed) 



MEAN 



S.D. 



16.314 


12.514 


13.064 


11.136 


35' 


35 
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APPENDIX 3 



TRIAL 3 DESCRIPTION 



Object 

To compare the post test scores and gain scores of 
students completing the C.A.L. biology course with 
the scores of students taught a similar course 
content in the same time period by the same teacher 
not using the C.A.L. system* 

Student Sample 

Seven schools were chosen ^ three grammar schools 
and four secondary modern schools. The teachers 
in each school taught two classes in the same year 
group. One class was to be the C*A,L. class, 
the other the non-C.A.L. class. Each group 

contained approximately 220 students. 

Experimental Design 

Before the course started students from each of the 
teacherfe two classes were matched on the following 
basisi^ 

Sex 

Teacher 
School 

Preknowledge of Subject 
Matter 

The teacher taught each class separately for five 
weeks. After this time both classes were tested 
for knowledge of subject matter. 

Results 

The students taught by the teacher using the C.A.L. 
system did significantly better on the post test 
and gain scores than the students taught^ by the same 
teacher without the C.A.L. system. ^ 
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TEST 7 


TEST 9 




N 




PRE 


POST 


GAIN 


PRE 


POST 


GAIN 


CAL 

CLASSES 


MEAN 


62.209 


73.548 


11. 339 


17.591 


35.139 


I7.54I 


115 


S.D. 


16.718 


16,566 


9 .202 


10.224 


17.245 


12.016 




NON- 
CAL 

CLASSES 


MEAN 


61. 809 


68. 791 


7.4 70 


18.313 


30.817 


12,678 


115 


S.D. 


16.646 


17. 786 


7.972 


10.082 


15.368 


10. 863 



* The difference between the CAL and NON-CAL 
means is sianif leant (5% level) 

+ The differunce between these gains is 
highly significant (1% level) 

MATCHED PAIRS 
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APPENDIX 4 

IMPLICATIONS OF DYNAMIC REALLOCATION OF OBJECTIVES 



The C*A,L. system as implemented can be best thought 
of as a series of nodes of a hypothetical network. 
Each node is a CALT plus its prerequisite plus 
its post test. 

Upon entry into the system, and on each occasion 
when the computer has to allocate the student a 
.. lesson, the computer searches the whole of its 
library (this search may be restricted but is 
not in general) for the next lesson. The choice 

of the next lesson is made on the basis of selecting 
the most complex lesson from the library of lessons 
which has prerequisites matched within defined 
limits to the students learning profile. Each 
lesson however has a different set of objectives. 
Thus, the system does not seek to ensure the 
student covers a specific subset of the total 
objectives of the course which could be predefined 
'^upon entry to the network but - rather attempts to 
ensure that each student has a reasonable chance 
of progressing successfully with the lesson. 
The general tendency of the system is to allocate 
students work which is likely to tax their ability* it will 
err on the side of being too difficult rather than 
too simple, 

A result of our experiments has been to show that 
instruction is really individualised in that no 
two students in Trial 1 followed the same course 
through the learning material. Thus, it can be 
argued thati^ 

a) The terminal objectives of the system 
cannot be predicted for any student upon 
entry* 

b) The terminal objectives for each student 
may (and probably will) be different. 

How then; is a system like this to be evaluated. 
Clearly evaluation is not just a statistical exercise,. 
In the circumstances in which this project has been 
conducted it has had to be a more broadly based 
exercise. As no norms exist in our C*A,L. system 
criterion reference techniques must be used but there 
is no single set ^6f criteria for all students. 

In the eyen± we have carried out all the standard 
statistical procedures and supplemented the information 
we have obtained by using student questionnaires and 
teacher interviews, wherever these have been appropriate. 

Ultimately, evaluation must produce for us a well 
documented picture of the system we have developed. 
The creation of this picture is one of our objectives* 
The picture will be printed v/ith all the tools we 
can use. We aim to make this as complete as possible. 



